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A study examining the effects of systematized 
feedback on matheniatics performance uas conducted in natural 
classroom settings to extend the external validity of previous 
feedback studies. A posttest-only, internal-external control group 
design tras used with three sixth grade classes serving as subjects. 
All classes utiliaed a **step** approach to math instruction, where 
students advanced through a pre*determined sequence of math skills at 
individual rates. Two classes, each taught by the same teacher, were 
used to meajsure treatment effects. One class served as an internal 
control grotp, receiving only math insti^uction. The other class 
received math instruction and systematized feedback. The remaining 
class, taught by a different teacher, served as an additional 
external control class. It was included to control for possible 
teacher bias and received only math instruction. At the end of 12 
weeks, all student:! were administered a 25*item test. A comparison of 
^:he scores of the three classes indicated that feedback is of 
significant value when applied in a systematic manner. The study 
confirmed the effectiveness and practicality of feedback based 
programs in ''-^eal world" settings. (Author/LLS) 
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ABSTRACT 

Feedback programs have produced inconsistent effects on student per- 
fortnance* The major contributing factors have been inconsistent definitio 
of feedback and feedback procedures used in various studies* Variations 
in the feedback recipients (students, teachers, parents) and a lack of 
careful utilization of the instructional content in the feedback systems 
have also contributed to the reported inconsistencies. In addition, feed" 
back programs have been criticized as being unrealistic in non- lab oratory 
settings* Many programs have not adequately considered the problems presented 
in typical classroom settings, and have imposed relatively complex feedback 
programs in contrived classrooms* Systematized feedback, a curriculuitb* based 
program employed In the present study, was defined as providing knowledge 
of results of student perfonnance to both students and teachers: a) related 
to established instructional objectives; b) within an established instruc" 
tional context; and c) on ^ regularly prescribed basis* 

A posttest-only, internal-external control group design was used in 
the present study* Three sixth grade classes served as sttbjects* All classes 
utilized a "step" approach to math instruction, where students advanced ^ 
through a p re-determined sequence of math skills at individual rates* Tvo 
classes, each taught by tlie same teacher, were used to measure treatment 
effects* One class served as an internal control group, receiving only 
math instruction* The other class received math instruction and system- 
atized feedback* The remaining class, taught by a different teacher, served 
as an additional external control class* The external control class, 
included to control for possible teacher bias, received only math instruction* 
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Tlie systematized feedback class was administered ? math skill inventory 
to dctennin& the individual skill status of each student. Student scores 
vere provided to the teacher, who subsequently distributed the results to 
the students. Students individually recorded their skills as loasteredi 
instructional, or not mastered on skill profile sheets. Students were 
instructed for a two veek period, and adioinistered an update quiz which 
covered skills not yet mastered* Student quizzes were scored, returned to 
the teacher for review, and subsequently distributed to the students for 
feedback and profile updating* This procedure continued for twelve weelcs* 
All students were administered a 25 item, math computation proficiency 
test at the end of the twelve week period* 

- Results of an ANOVA comparing the scores of the three classes 
indicated significant differences (p<*02)* Means and standard deviations 
for the three classes were: systematized feedback 0E='21*2p; S*D*=3*D7), 
internal control (X'^lS^TSi S.D**-3.67), and external control (X*=18*6l; 
S*D*==3*47)* The systematiEed feedback class scored significantly higher, 
than the control groups on the math proficiency test, while no significant 
differences were obtained between the two control groups* The observed 
treatment effect and the non-signlf leant differences between the control 
classes suggests that the results are valid, and not simply a function of 
teacher bias* 

Feedback is of significant value when applied in a systematic manner* 
The present study confirms the effectiveness and practicality of feedback 
based programs in "real world" settings* Feedback programs should system- 
atically incli!de instructional content, feedback procedures that include 
both teachers and students, and provisions for assuring compatability 
with existing instructional settings. 



SySTEHATIEED FEEDBACK AND MATHEMATICS PERFORMANCE 

Michael J. Hannafin 
Arizona State University 

^e benefits of providing cielayed feedback .in the i nstructional 
process have been well documented in controlled settings (Sturges^ Sarafino 
and Donaldson^ 1968; More, 1969; Kulhavy, 1972; Sassenrath, 1975). Yet 
ccMnparatively little research demonstrating applicability of delayed feed- 
bade in natural classroom settings has been reported. Limited attempts to 
apply delayed feedback programs in natural classroom settings have often 
produced contradictory results {e.g., Surber & Anderson* 1975, versus New- 
man, Williams, & Hiller, 1974)* Hie determination of the practical effec- 
tiveness of delayed feedback in natural classroom settings is imperative. 

TOie present study examined the effects of providing delayed, system- 
atized feedback on student test scores on math computation proficiency 
tests. Delayed systematized feedback was defined as providing knowledge 
of test results on skill quizzes to both students and teachers: a) related 
to established instructional objectives; b) within an established instruc- 
tional context, i^e. , natural classroom setting; c) on a bi-weekly 
basis} and d) .one day following the cotiipletion of the <juiz. 

Several in^rtant components of delayed feedback programs have been 
identified. Sturges (1972 a) suggested that the effecitveness of delayed 
feedback programs is contingent on the nature of the stimuli present during 
the feedback, how the students oi^ subjects respond to the feedbacks and 
the relevance of the feedback stimuli to the test measures* KUlhavy (1977) 
noted that although feedback should be used frequently in the instructional 
process, the availability of feedback materials needs to be restricted. 



He reasoned tliat when feedback materials are too readily available^ 
students might copy responses rather than utilising the correcting function 
of feedback to confirm the accuracy of their responses. Kulhavy also 
suggested that the instructional level uponvhich the feedback is based 
must be appropriate for the intended learners* Feedback systems produce 
only minimal effects when students have very little confidence in their 
responses. Cqn':equently^ the required learning tasks must be reasonably 
attainable for delayed feedback strategies to be effective (Kulhavy, 1977). 
Sturges (1972 b) suggested that the activity immediately following feed- 
back is also critical in feedback programs. Given opportunities for self- 
correction or practice following feedback, students' performance was 
improved as a result of the feedback (Sturges, 1972 b) . In effect, the 
identified components suggest that feedback must be systematically applied 
in order to be maximally effective. 

U'he manner in which delayed feedback is operationally defined is also 
an important factor. Delayed feedback has been frequently defined as 
providing knowledge of results of test performance to students. However, 
the different ways in which such definitions have been operational i:£ed have 
been a source of concern. Kulhavy (1977) attributed many of the reported 
inconsistencies among delayed feedback programs to operational definitional 
differences. 

The manner in which knowledge of results is provided has also been 
consicered an additional source of potential variability. Sturges (1969) 
noted that knowledge of results in delayed feedback programs should bo 
informative, i.e., must include information related to criterion measure 
items and response alternatives. Gilman (1969) has suggested that ti\e 
process of providing knowledge of results be modified to provide greater 



guidance to the learners. The author found that feedback which simply 
identifies responses as correct or incorrect was not as effective as 
providing knowledge of results with guided elaboration. Guided elaboration, 
where both correct and incorrect responses are verified through a structured 
review of the relevant content, was more effective in improving student 
performance than providing only knowledge of response accuracy (oilman , 
1969) . 

iUi additional definitional problem reported in feedback literature is 
related to the immediate vs. delayed feedback dichotomy. Since the terms 
iramediat e and delayed are relative terms, they have been operationally 
defined within each individual study. Consequently, the generalizability 
of research findings has been somewhat limited (Peck £« Tillema, 1979, 
Note 1). 

Attempts have also been made to identify maximally effective delay 
intervals as applied in delayed feedback programs. Studies employing 
delayed feedback techniques have focused primarily on retention of factual 
Information. Research in written prose learning generally indicates that 
feedback delays from one-to-two days are effective in improving subsequent 
student performance. English and Kinzer (1966) found that feedback delays 
of one hour or two days were superior to either immediate feedback or feed-^ 
back delayed one week. More (1969) found that delays of two^and-one-half 
hours or one day were superior to either inmediate feedback or feedback 
delayed four days. However, the primary means for assessing student perfor- 
mance in these studies has been the use of multiple choice tests. 
effectiveness of feedback programs in process-oriented content areas 
has not been well documented. Computational mathematics, for 
example, require*^ both basic factual memory and procoss-oriented applica- 
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tions. Existing feedback research does not provide clear guidelines for the 
development of feedback programs process-* oriented content areas such as 
mathematics. 

The use of feedback systems in relearning information is also an 
important consideration. However, the manner in which feedback informci*:ion 
should be stibsequently utilized has not been clearly established. While 
this is critical to the development of an effective feedback program, 
research has not conclusively identified how feedback information should 
be subsequently utilized in order to be maximally effective. Surprisingly 
little research has been reported pertaining to the role of the instructor 
in teacher-directed learning settings. In such settings, the teacher typically 
controls the content, presentation rate, and total time provided for 
instruction. Since these are potent variables in facilitating student 
learning, the role of the teacher needs to be investigated. 



Method 

Subjects 

Hie stibjects were 67 students assigned to one of three sixth grade 
mathematics classes in a non-tracked stiburban elementary school. Non- 
tracked refers to the essentially random class assignment procedures used 
by the school system. The three classes were selected because each used a 
"step" approach tomathematics instruction, where students ad^ranced through 
a predetermined sequence of mathematics skills at individual rates^ ivo 
classes were taught by the same teacher, and the r^tnainlng class was taught 
by a second teacher. One class was the syettematized feedback 9i^up, one the 
internal control group, and one the external control group. 



1, 
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Materials 

The materials used in the present study included a mathematics skill 
inventory, a series of sequenced mini-^quizzcs , student profile sheets for 
recording individual progress, and class record sheets for reporting 
overall class progress* l^e mathematics inventory included three test 
items for each of 39 specific objectives* objectives were developed 

and sequence by school personnel prior to the present study and the 
mathematics inventory was developed by the researcher for the present study* 
llie inventory provided information tised to establish the baseline skill 
status for each student in the systematized feedback group* ^e mathematics 
inventory also served as the initial feedback instrument for both teachers 
and students regarding individual skill acquisition* ^irty-nine separate 
mini^quizzes, each keyed to the mathematics ob;}ectives, were used for a bi" 
' weekly skill assessment of students in the systematized feedback group* 
Student performance on the mini^quizzes was the primary information source 
for providing the bi-weekly feedback* In addition, each student in the 
systematized feedback group maintained an individual skill profile sheet 
which included the 39 math objectives* Students individually recorded their 
skills as mastered (100% accuracy), instructional (67% accuracy), or not 
mastered (less ,than 67% accuracy) on their profile sheets* The skills were 
sequentially ordered from easiest to most difficult based upon the skill 
sequence defined by the school personnel* The individual skill profile 
sheets provided students with the means to monitor their individual progress 
throughout the study* Based upon individual student performance on the 
bi-weekly mini-quizzes, students individually updated their profiles* A 
class summary record sheet, which provided^a student performance by objec^ 
tive format, provided a capsulized summary of student performance and 
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progress throughout the study* 
Criterion Measures 

The criterion test used in the present study was a 25 item mathematics 
computation test. The computation test included skills which are typically 
inastered between fifth and eighth grade levels^ and all skills on the 
computation test were also included on the mathematics inventory and mini- 
quizzes. The test required addition, stibtraction, multiplication, and 
division of whole nxunbers, fractions, and decimals* In addition, test 
items related to physical geometry and measurement were included* The 
test, which was developed by the school staff, was administered to provide . 
information for the present study ancl information for use in subsequent 
class scheduling. The test was a constmcted response test, and was 
administered as a power test. 
Procedures 

Three classes participated in the study. TWo classes were taught by 

*" ' * * * . 

the same teacher, the remaining class was taught by a different teacher* 

Of the two classes taught by the same teacher, one class was randomly 
assigned to the systematized feedback treatment. The remaining classes 
served as control classes. 

Prior to the start of the study, the systematized feedback teacher 
provided students with a general introduction to the niathematic program that 
followed* No statement was made regarding comparisons among classes; 
students were not informed that their test scores would be compared with 
the performance of other classes. Students were administered the mathematic 
inventory under untimed conditions prior to the start of the program. Math- 
ematics inventories were scored, student performance was recorded on the 
class record sheet and student profile sheets, and all information was 
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returned to the teacher on the first day of the program, ^e teachsr 
distributed the student profile sheets to the class with the corresponding 
scored mathematics inventory test. At that time, the teacher conductsd a 15 
minute orientation session designed to familarize the students with the 
relationship between their test performance and their individual profile 
sheets, ^e teacher provided instructions to the students for recording 
their test scores during the subsequent feedback periods provided during 
the study, ah students then received instruction for a two week period. 
At the end of the two week period, students in the syst^natizod feedback 
class were administered mini-quizzes covering those skills not yet mastered. 
Since many students had not yst been exposed to several of the more difficult 
skills, they were instructed to review each test item, write their answers 
if they felt reasonably confident they could perform the task, or mark an 
in the answer box if they were uncertain of the correct response. The 
mini-quizzes were corrected, scored, and returned to the teadher and the 
students on the next school day. At that time, the teacher distributed 
the scored mini-quizzes to the students and instructed them to update their 
profile sheets according to the quiz results. Students were provided 
approximately five minutes for completing this task, ^e teacher then 
instructed the students to review their correct and incorrect answers in 
order to identify possible problems. Students were told to request assis** 
tance from the teacher, if needed, or to proceed with the next step in the 
mathematic sequence. Since students in all classes were routinely instructed 
to request needed assistance, this procedure simply co;, -rmed the availability 
of the teacher under the new program. 

Ttie bi-weekly testing, scoring, and feedback was continued throughout 
a twelve week period. At the end of the twelve week period, students in 



both the systematized feedba^'^k class and the control classes were 
administered the 25 item mathematics computation test. 



■ Results 

Means and standard deviations for the mathematics computation test scores 
by treatment group are included in Table 1. 



Insert Table 1 Here 



As shown in Table 1# the systematized feedback class scored higher than 
either the internal control class or the external control class. Results 
of a one*-way ANOVA indicated the difference to be significant (F^4. 18»_df =* 
2/64| £<.02). Ko significant differences were obtained between the two 
^ntrol groups. 

Discussion 

The present study examined the effects of systematized feedback on 
mathematics performance. The study was conducted in natural classroom set- 
tings in an attempt to extend the external validity of previous feedback 
studies. 

^e results of the present study indicate that feedback can be of 
significant valu& when applied in a systematic manne-r. All classes employed 
in the present study employed the same instructional materials, approximately 
the same student entry level skills based on teacher and administrator report, 



Table 1 

Mean Computation Test Scores, H, and Standard 
Deviations by Treatment Group 



Treatment Group 



Cbmputation Test Systematized Internal External 

Hesults Feedback Control Control 



Test Score 


■ 21.20* 


18.75 


18. 61 


S.D. 


3.07 


3.67 


3.49 


H 


24 


20 * 


23 



*E<.02 
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and the same performance documentation system required by the school, 
major departures among the classes * tployed in the present study were that 
the systematized feedback class employed an instructionally integrated feed- 
back system. Systematized feedback was regularly provided, organized, 
included provisions for post-feedback instruction, and was provided to both 
students and teachers. 

In effect, systematized feedback provides a closed loop instructional 
system where knowledge of student performance is sensibly integrated with 
instruction and assessment to improve subsequent student performance. Since 
the major difference between the systematized feedback class and the control 
classes was essentially organizational in nature, the findings suggest that 
a more methodical approach to providing instruction in natural settings ia 
both plausible and effective. 

During the present study, no special requirements were made of the 
systematized feedback teacher regarding instructional style, teacher 
was instructed to use the information in whatever manner deemed appropriate. 
However, *'S verified informally by the teacher, teaching stiyle was modified 
as a result of the systematized feedback received during the study. TOie 
systematized feedback teacher indicated that the instruction became more 
focused, i.e., more skill-specific. Also, the teacher noted that student 
progress became contingent on demonstrable skill acquisition rather than 
teacher judgement. While it is impossible to partition the proportion of 
score variance accounted for by student versus teacher knowledge of results 
in the present study, the combined effect is apparent. When a teacher was 
provided meaningful information upon which to base modifications in teaching 
style, changes occurred; when such information was provided on a regular 
basis, as demonstrated during the present study, the teacher modified 
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instructional strategies on an on- going basis, result of this type of 

on-going instructional modification was inevitably improved student 
performance. Since the teactier possesses substantial control over the 
instructional process, the importance of including the teacher in delayed 
feedback programs can not be overstated. While the student is ultimately 
responsible for learning the infonuation presented, the teacher typically 
controls what and how information is to be presented. 

Several cautions which are frequently encountered in applied, action 
research must be considered. Although random assignment of students to 
classes was assumed no absolute statements of pre*-program equivalence among 
classes can be guaranteed. Historically, however, substantial mathematics 
performance differences among sixth grade classes was considered very 
uncommon by the school administration. Also, since the researcher randomly 
assigned the systematized feedback class, potential teacher or subject selec- 
tion bias was not a factor. Future research should systematically account 
for equivalence through pretest procedures or covariance techniques. 

In some respects, the strengths of the present study might be perceived 
as the weaknesses of a more rigidly controlled study. In the present study, 
however, the exteimal validity of the controlled delayed feedback research 
was of particular concern. The present study confirms the effectiveness 
and practicality of feedback based programs in "real world" settings. 

Additional research regarding the generalizability of delayed feedback 
research to other prDcess*^oriented subject areas should be advanced. 
Although computational mathematics is largely process in nature f the present 
study utilised a falrly'Well articulated instructional system. It is unclear 
whether or not academic stibjects involving literature or creative writing 
are equally amenable to a systematized feedback approach. ^ 
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An additional area for further research concerns the effects of 
systematized feedback on task-relevant student questioning. It :.s possible 
that systematized feedback increases the number of task-relevant questions 
due to the information available for student review. However, it is also 
possible that student questionning wuld l>e reduced as a function of the 
preciseness and task relevance of the information provided during systematized 
feedback. Such possibilities should be investigated. 

In summary, feedback is of significant value when applied in a systemtic 
manner. TOie present study confirms the effectiveness and practicality of- 
systematically aj^lied feedback in natural classroom settings. Feedback 
programs should systematically include instructional content, feedback 
procedures that include both teachers and students, and provision for 
assuring cotnpatability within the instructional setting. 
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